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R. Abbasi,23 Y. Abdou,i^ M. Ackermann,^^ J. Adams," M. Ahlers,^^ K. Andeen,23 J. AuflFenberg,^^ X. Baij^^ 
. Baker,23 S. W. Barwick/^ R. Bay/ J. L. Bazo Alba/s K. Beattie,*^ S. Bechet/" J. K. Becker/^ K.-H. Becker/-* 

M. L. Benabderrahmane/''^ J. Berdermann/^ P. Berghaus/^ D. Berley/^ E. Bernardini/^ D. Bertrand/° 
D. Z. Besson/i M. Bissok/ E. Blaufuss/'' D. J. Boersma/^ C. Bohm/^ J. Bolmont/^ S. Boser/^ O. Botner/^ 
. Bradley/1 J. Braun/^ D. Breder/'^ T. Burgess/^ T. Castermans/^ D. Chirkin/^ B. Christy/'' J. Clem/^ 
Cohen/° D. F. Cowen/^'^o M. V. D'Agostino/ M. Danninger/^ C. T. Day/ C. De Clercq/i L. Demirors/" 
O. Depaepe/i F. Descamps/"^ P. Desiati/^ G. de Vries-Uiterweerd/''' T. DeYoung/* J. C. Diaz-Velez/^ 
J. Drcyer/6 J. P. Dumm/^ M. R. Duvoort/^ W. R. Edwards/ R. Ehrlich/"* J. Eisch/^ R. W. Ellsworth/'' 
Engdegard/2 S. Euler/ P. A. Evenson/'^ O. Fadiran/ A. R. Fazely/ T. Feusels/^ K. Fihmonov/ C. Finley/^ 
1. M. Foerster/i B. D. Fox/* A. Franckowiak/ R. Franke/^ T. K. Gaisser/*^ J. Gallagher/^ R. Ganugapati/^ 
L. Gerhardt/'^ L. Gladstone/^ A. Goldschmidt/ J. A. Goodman/'' R. Gozzini/^ D. Grant/* T. Griesel/^ 
A. Grofi/3'18 S. Grullon/3 R. M. Gunasingha/ M. Gurtner/" C. Ha/* A. Hallgren/^ F. Halzen/^ K. Han/^ 
K. Hanson/3 Y. Hasegawa/^ J. Heise/^ K. Helbing/" P. Herquet/^ S. Hickford/^ G. C. HiU/^ K. D. Hoffman/" 
K. Hoshina/3 D. Hubert/^ W. Huelsnitz/'* J.-P. Hulfi/ P. O. Hulth/^ K. Hultqvist/^ S. Hussain/*^ 
R. L. Imlay/ M. Inaba/^ A. Ishihara/^ J. Jacobsen/^ G. S. Japaridze/ H. Johansson/^ J. M. Joseph/ 
K.-H. Kampert/" A. Kappes/^-Q T. Karg/" A. Karle/^ J. L. Kelley/3'0 P. Kenny/* J. Kiryluk/-^ F. Kislat/^ 
S. R. Klein/' ^ S. Klepser/^ S. Knops/ G. Kohnen/^ H. Kolanoski/ L. Kopke/" M. Kowalski/ T. Kowarik/" 
M. Krasberg/3 K. Kuehn/^ T. Kuwabara/^ M. Labare/° K. Laihem/ H. Landsman/^ R. Lauer/^ 
H. Leich/^ D. Lennarz/ A. Lucke/ J. Lundberg/^ J. Lunemann/'* J. Madsen/* P. Majumdar/^ 
R. Maruyama/3 K. Mase/^ H. S. Matis/ C. P. McParland/ K. Meagher " M. Merck/3 P. Meszaros/^-^* 
E. Middell/5 N. Milke/6 H. Miyamoto/^ A. Mohr/ T. Montaruh/3'0 R. Morse/3 S. M. Movit/" 
K. Miinich/6 R. Nahnhauer/5 J. W. Nam/^ P. Niefien/^ D. R. Nygren/-^^ S. Odrowski/^ A. Olivas/" 

M. Olivo/2 M. Ono/^ S. Panknin/ S. Patton/ C. Perez de los Heros/^ J. Petrovic/*' A. Piegsa/" 
D. Pieloth/5 A. C. Pohl/2.[l R. Porrata/ N. Potthoff/" P. B. Price/ M. Prikockis/* G. T. Przybylski/ 
. Rawhns/ P. Redl/'* E. Resconi/^ W. Rhode/*' M. Ribordy/° A. Rizzo/* J. P. Rodrigues/^ P. Roth/" 
F. Rothmaicr/" C. Rott/^ C. Roucelle/^ D. Rutledge/* D. Ryckbosch/^ H.-G. Sander/" S. Sarkar/^ 

K SafalpfVa 35 c QpVilpnstpHf 35 T SrhmlHt 14 n SrVinplHpr 23 A Qphnkraft 1 O Srhiil?. 18 M SrhnnrV 1 



F. Rothmaicr/" C. Rott/^ C. Roucelle/^ D. Rutledge/^ D. Ryckbosch/^ H.-G. Sander/" S. Sarkar/^ 
K. Satalecka/5 S. Schlenstedt/^ T. Schmidt/" D. Schneider/3 A. Schukraft/ O. Schulz/^ M. Schunck/ 

D. Seckel/*' B. Semburg/" S. H. Seo/^ Y. Sestayo/^ S. Seunarine/^ A. Silvestri/^ A. Shpak/^ G. M. Spiczak/^ 
C. Spiering/5 T. Stanev/*' G. Stephens/^ T. Stezelberger/ R. G. Stokstad/ M. C. Stoufer/ S. Stoyanov/^ 

E. A. Strahler/3 T. Straszheim/" K.-H. Sulanke/^ G. W. Sullivan/" Q. SwiUens/"^ I. Taboada/ O. Tarasova/^ 
A. Tepe/" S. Ter-Antonyan/ C. Terranova/° S. Tilav/^ M. Tluczykont/^ P. A. Toale/" D. Tosi/^ D. Turcan/" 

N. van Eijndhoven/^ J. Vandenbroucke/ A. Van Overloop/"^ B. Voigt/^ C. Walck/^ T. Waldenmaier/ 
M. Walter/5 C. Wendt/^ S. Westerhoff/3 N. Whitehorn/3 C. H. Wiebusch/ A. Wiedemann/^ G. Wikstrom/^ 
D. R. WiUiams/ R. Wischnewski/^ H. Wissing/-"" K. Woschnagg/ X. W. Xu/ G. Yodh/^ and S. Yoshida^^ 



van tijndhoven, J. Vandenbroucke, A. Van Uverloop, a. Voigt, O. Walck, i. Waldenmaier, 
lter,35 C. Wendt,23 S. Westerhoff,23 N. Whitehorn,23 C. H. Wiebusch," A. Wiedemann,"*^ G. Wikstrom,^^ 
Wimams,2 R. Wischnewski,35 H. Wissing,!-"" K. Woschnagg,^ X. W. Xu,^ G. Yodh,"^ and S. Yoshida^^ 

(The IceCube Collaboration) '0 

III Physikalisches Institut, RWTH Aachen University, D-52056 Aachen, Gen 
Dept. of Physics and Astronomy, University of Alabama, Tuscaloosa, AL 3548' 
"'Dept. of Physics and Astronomy, University of Alaska Anchorage, 
3211 Providence Dr., Anchorage, AK 99508, USA 
^CTSPS, Clark-Atlanta University, Atlanta, GA 30314, USA 
^School of Physics and Center for Relativistic Astrophysics, 
Georgia Institute of Technology, Atlanta, GA 30332, USA 
^Dept. of Physics, Southern University, Baton Rouge, LA 70813, USA 
^Dept. of Physics, University of California, Berkeley, GA 94720, USA 
^Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA 
''institut fiir Physik, Humboldt- Universitdt zu Berlin, D- 12489 Berlin, Germany 
Universite Libre de Bruxelles, Science Faculty CP230, B-1050 Brussels, Belgium 
Vrije Universiteit Brussel, Dienst ELEM, B-1050 Brussels, Belgium 
^^Dept. of Physics, Chiba University, Chiba 263-8522, Japan 
'Dept. of Physics and Astronomy, University of Canterbury, Private Bag 4800, Christchurch, New Zealand 
'''Dept. of Physics, University of Maryland, College Park, MD 20742, USA 
'^Dept. of Physics and Center for Cosmology and Astro- Particle Physics, 
)hio State University, 191 W. Woodruff Ave., Columbus, OH 43210, USA 



IS 

lyepi. uj r iiy 

H 
15 



2 



^^Dept. of Physics, TU Dortmund University, D-44221 Dortmund, Germany 
'^Dept. of Subatomic and Radiation Physics, University of Gent, B-9000 Gent, Belgium 
Max-Planck-Institut fiir Kernphysik, D-69177 Heidelberg, Germany 
^^Dept. of Physics and Astronomy, University of Galifornia, Irvine, GA 92697, USA 
^"Laboratory for High Energy Physics, Ecole Polytechmque Federale, GH-1015 Lausanne, Switzerland 
'^'^ Dept. of Physics and Astronomy, University of Kansas, Lawrence, KS 66045, USA 
^^Dept. of Astronomy, University of Wisconsin, Madison, WI 53706, USA 
^'^ Dept. of Physics, University of Wisconsin, Madison, WI 53706, USA 
^'^ Institute of Physics, University of Mainz, Staudinger Weg 7, D-55099 Mainz, Germany 
^'''University of Mons-Hainaut, 7000 Mons, Belgium 
'^^Bartol Research Institute and Department of Physics and Astronomy, 
University of Delaware, Newark, DE 19716, USA 
^''Dept. of Physics, University of Oxford, 1 Kehle Road, Oxford 0X1 3NP, UK 
^^Dept. of Physics, University of Wisconsin, River Falls, WI 54022, USA 
Dept. of Physics, Stockholm University, SE- 10691 Stockholm, Sweden 
''Dept. of Astronomy and Astrophysics, Pennsylvania State University, University Park, PA 16802, USA 
Dept. of Physics, Pennsylvania State University, University Park, PA 16802, USA 
'^Dept. of Physics and Astronomy, Uppsala University, Box 516, S-75120 Uppsala, Sweden 
'^'^Dept. of Physics and Astronomy, Utrecht University/SRON, NL-3584 GG Utrecht, The Netherlands 
'^'^Dept. of Physics, University of Wuppertal, D-42119 Wuppertal, Germany 
■'^DESY, D-15735 Zeuthen, Germany 
(Dated: June 2, 2009) 

The AMANDA-II detector, operating since 2000 in the deep ice at the geographic South Pole, 
has accumulated a large sample of atmospheric muon neutrinos in the 100 GeV to 10 TeV energy 
range. The zenith angle and energy distribution of these events can be used to search for various 
phenomenological signatures of quantum gravity in the neutrino sector, such as violation of Lorentz 
invariance (VLI) or quantum decoherence (QD). Analyzing a set of 5511 candidate neutrino events 
collected during 1387 days of livetime from 2000 to 2006, we find no evidence for such effects 
and set upper limits on VLI and QD parameters using a maximum likelihood method. Given the 
absence of evidence for new flavor-changing physics, we use the same methodology to determine the 
conventional atmospheric muon neutrino flux above 100 GeV. 

PACS numbers: 95.55.Vj, 14.60.St, ll.30.Cp, 03.65.Yz, 04.60.-m 



I. INTRODUCTION 

Experimental searches for possible low-energy signa- 
tures of quantum gravity (QG) can provide a valuable 
connection to a Planck-scale theory. Numerous quantum 
gravity theories suggest that Lorentz invariance may be 
violated or spontaneously broken, including loop quan- 
tum gravity [l| , noncommutative geometry , and string 
theory This, in turn, has encouraged phenomeno- 
logical developments and experimental searches for such 
effects [1, [1] . Space-time may also exhibit a "foamy" na- 
ture at the smallest length scales, inducing decoherence 
of pure quantum states to mixed states during propaga- 
tion through this background Q. 

The neutrino sector is a promising place to search for 
such phenomena. Neutrino oscillations act as a quantum 
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interferometer, and QG effects that are expected to be 
small at energies below the Planck scale can be amplified 
into large flavor-changing signatures. Water-based or ice- 
based Cherenkov neutrino detectors such as BAIKAL , 
AMANDA-II [H, ANTARES [lH, and IceCube [H have 
the potential to accumulate large samples of high energy 
atmospheric muon neutrinos. We present here an analy- 
sis of AMANDA-II atmospheric muon neutrinos collected 
from 2000 to 2006 in which we search for flavor-changing 
signatures that might arise from QG phenomena. 

In addition to searches for physics beyond the Stan- 
dard Model, a measurement of the conventional atmo- 
spheric neutrino flux is useful in its own right. Uncer- 
tainties in the incident primary cosmic ray spectrum and 
in the high energy hadronic interactions affect the atmo- 
spheric neutrino flux calculations (see e.g. Refs. 
Atmospheric neutrinos are the primary background to 
searches for astrophysical neutrino point sources and dif- 
fuse fluxes, so knowledge of the flux at higher energies 
is crucial. In this analysis, we vary the normalization 
and spectral index of existing models for the atmospheric 
neutrino flux and determine the best-fit spectrum. 

We begin with a review of the phenomenology relevant 
to our search for new physics in atmospheric neutrinos. 
Next, we describe the AMANDA-II detector, data selec- 
tion procedures, and atmospheric neutrino simulation. 
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Third, we describe the analysis methodology by which 
we quantify any deviation from conventional physics. We 
do not observe any such deviation, and hence we present 
upper limits on violation of Lorentz invariance (VLI) and 
quantum decoherence (QD) obtained with this method- 
ology, as well as a determination of the conventional at- 
mospheric neutrino flux. 



II. PHENOMENOLOGY 



A. Atmospheric Neutrinos 

Atmospheric neutrinos are produced when high energy 
cosmic rays collide with air molecules, producing charged 
pions and kaons that subsequently decay into muons and 
muon neutrinos. Observations of atmospheric neutrinos 
by Super-Kamiokande 0, Soudan 2 MACRO [13, 
and other experiments have provided strong evidence for 
mass-induced atmospheric neutrino oscillations. The re- 
lationship between the mass eigenstates and the flavor 
eigenstates can be characterized by three mixing angles, 
two mass splittings, and a complex phase. Because of the 
smallness of the 613 mixing angle and the Ami2 splitting 
(see Ref. [18] for a review), it suffices to consider a two- 
neutrino system in the atmospheric case, and the survival 
probability for muon neutrinos of energy E as they travel 
over a baseline L from the production point in the atmo- 
sphere to a detector is 



have Lorentz boosts exceeding 10'^"'^ and provide a sensi- 
tive test of Lorentz symmetry. 

Neutrino oscillations in particular provide a sensitive 
testbed for such effects. Oscillations act as a "quantum 
interferometer" by magnifying small differences in energy 
into large flavor changes as the neutrinos propagate. In 
conventional oscillations, this energy shift results from 
the small differences in mass among the eigenstates, but 
specific manifestations of VLI can also result in energy 
shifts that can generate neutrino oscillations with differ- 
ent energy dependencies. 

In particular, we consider VLI in which neutrinos have 
limiting velocities other than the canonical speed of light 
c ([1,13]; see the appendix for further background). Since 
these velocity eigenstates can be distinct from the mass or 
flavor eigenstates, in a two-flavor system this introduces 
another mixing angle ^ and a phase 77. The magnitude of 
the VLI is characterized by the velocity-splitting between 
the eigenstates, Ac/c = (cai — Ca2)/c. 

In this form of VLI, the survival probability is [T^ 



P..^.. = 1 - sin^ 29 sin^ { 7^ ) , (2) 



4:E 



where the combined effective mixing angle O can be writ- 
ten 



sin^ 26 = ^ (sin^ 2e+R^ sin^ 2^-^2iT! sin 29 sin 2^ cos 77) , 
7^^ 



sin^ 26^ 
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(1) 



the correction to the oscillation wavelength TZ is 



(3) 



where L is in inverse energy units (we continue this con- 
vention unless noted otherwise). In practice, the zenith 
angle of the neutrino serves as a proxy for the baseline 
L. 

A recent global flt to oscillation data results in best-fit 
atmospheric oscillation parameters of Aml^^ — 2.39 x 
10-3 eV^ and sin^ 2e^tm = 0.995 pj]. Thus, for energies 
above about 50 GeV, atmospheric neutrino oscillations 
cease for Earth-diameter baselines. However, a number 
of phenomcnological models of physics beyond the Stan- 
dard Model predict flavor-changing effects at higher en- 
ergies that can alter the zenith angle distribution and 
energy spectrum of atmospheric muon neutrinos. We re- 
view two of these here, violation of Lorentz invariance 
and quantum decoherence. 



B. Violation of Lorentz Invariance 

Many models of quantum gravity suggest that Lorentz 
symmetry may not be exact [H] . Even if a QG theory is 
Lorentz symmetric, the symmetry may still be sponta- 
neously broken in our Universe. Atmospheric neutrinos, 
with energies above 100 GeV and mass less than 1 eV, 



TZ = + + 2i?(cos 29 cos 2^ + sin 29 sin 2^ cos rj) , 

(4) 

and the ratio R between the VLI oscillation wavelength 
and mass-induced wavelength is 



R 



AcE AE 
c 2 Am2 



(5) 



for a muon neutrino of energy E and traveling over 
baseline L. For atmospheric neutrinos, we flx the con- 
ventional mixing angle 9 = ^atm and mass difference 
Am? = Am^tin to the global fit values determined in 
Ref. [20I of Aml^^ = 2.2 x lO'^ eV^ and sin^ 26latm ^ 1- 
For simplicity, the phase is often set to or 7r/2. For 
illustration, if we take both conventional and VLI mixing 
to be maximal (^ = 9 = tt/A), this reduces to 



Py^^^y^ (maximal) = 1 — sin^ 



Am^L 

AE 



Ac LE 
~c 2~ 



(6) 

Note the different energy dependence of the two effects. 
The survival probability for maximal baselines as a func- 
tion of neutrino energy is shown in Fig. [TJ 
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VLI oscillation length L oc E ^ to other integral powers 
of the neutrino energy E, that is, 



4 5 5 
log^^ / GeV 



FIG. 1: ffi survival probability as a function of neutrino 
energy for maximal baselines (L ~ 2i?Earth) given conven- 
tional oscillations (solid line), VLI (dotted line, with n = 1, 
sin 2^ = 1, and AS — 10"'^^), and QD effects (dashed line, 
with n = 2 and D* = 10"^° GeV"^). 



Several neutrino experiments have set upper limits on 
this manifestation of VLI, including MACRO [l^l, Super- 
Kamiokande [2^, and a combined analysis of K2K psj 
and Super-Kamiokande data [3 (Ac/c < 2.0 x 10~^^ 
at the 90% CL for maximal mixing). In previous work, 
AMANDA-II has set a preliminary upper limit using four 
years of data of 5.3 x 10~^^ Other neutrino tele- 

scopes, such as ANTARES, are also expected to be sen- 
sitive to such effects (see e.g. ReL pH). 

Given the specificity of this particular model of VLI, 
we wish to generalize the oscillation probability in Eq. [51 
We follow the approach in [25| , which is to generalize the 



Ac LE 
~c 2~ 



A5- 



(7) 



where n G {1, 2, 3}, and the generalized VLI term A(5 is in 
units of GeV""+^ An L oc E'^ energy dependence (n = 
2) has been proposed in the context of loop quantum 
gravity [26| and in the case of non-renormalizable VLI 
effects caused by the space-time foam [l^] . Both the L oc 
E^^ {n = 1) and the L oc E~'^ {n ~ 3) cases have been 
examined in the context of violations of the equivalence 
principle (VEP) [2^,[2^,[3^. In general, Lorentz violation 
implies violation of the equivalence principle, so searches 
for either effect are related fE]. 

C. Quantum Decoherence 

Another possible low-energy signature of QG is the 
evolution of pure states to mixed states via interaction 
with the environment of space-time itself, or quantum 
decoherence. One heuristic picture of this phenomenon 
is the production of virtual black hole pairs in a "foamy" 
spacetime, created from the vacuum at scales near the 
Planck length [3l|. Interactions with the virtual black 
holes may not preserve certain quantum numbers like 
neutrino flavor, causing decoherence into a superposition 
of flavors. 

Quantum decoherence can be treated phenomenologi- 
cally as a quantum open system that evolves thermody- 
namically (we refer the reader to the appendix for more 
detail). In a three-flavor neutrino system, the decoher- 
ence from one flavor state to a superposition of flavors can 
be characterized by a set of parameters Di,i G {1, . . . , 8} 
that represent a characteristic inverse length scale over 
which the decoherence sets in. The survival probabil- 
ity in such a system is [s^l 



- + - -e-^"Hl + coa2e)^ + —e~^"<' (1-3 00826)^ 
3 2 4 12 



(8) 



COS 



Am2 
E 



+ 



(De - Dry 



\l{^f-{D,-D,) 



(De-Dr)^ 
J 



Note the limiting probability of 1/3, representing full de- 
coherence into an equal superposition of flavors. The Di 
not appearing in Eq. [8] affect decoherence between other 
flavors, but not the survival probability. 



the VLI effects, we choose a generalized phenomenolog- 
ical approach where we suppose the Di vary as some 
integral power of the energy, that is 



The energy dependence of the decoherence terms Di 
depends on the underlying microscopic model. As with 



D, = D*E'\ ne{l,2,3} 



(9) 
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where E is the neutrino energy in GeV, and the units 
of the D* are GeV"""*"^. The particularly interesting 
E'^ form is suggested by decoherence calculations in non- 
critical string theories involving recoiling D-brane geome- 
tries [sj]. We show the n = 2 survival probability as 
a function of neutrino energy for maximal baselines in 

Fig.m 

An analysis of Supcr-Kamiokande in a two-flavor 
framework has resulted in an upper limit at the 90% CL 
of D* < 9.0 X 10-28 GeV"^ for an E"^ model and all D* 
equal [s^. ANTARES has reported sensitivity to vari- 
ous two- flavor decoherence scenarios as well, using a more 
general formulation [sl]. Analyses of Super-Kamiokande, 
KamLAND, and K2K data [s^, HI] have also set strong 
limits on decoherence effects proportional to i?" and E~^. 
Because for such effects our higher energy range does not 
benefit us, we do not expect to be able to improve upon 
these limits, and we focus on effects with n > 1. 



III. DATA AND SIMULATION 
A. The AMANDA-II Detector 

The AMANDA-II detector consists of 677 optical mod- 
ules (OMs) on 19 vertical cables or "strings" frozen into 
the deep, clear ice near the geographic South Pole. Each 
OM consists of a 20 cm diameter photomultiplier tube 
(PMT) housed in a glass pressure sphere. Chcrcnkov 
photons produced by charged particles moving through 
the ice trigger the PMTs. Combining the photon arrival 
times with knowledge of the absorption and scattering 
properties of the ice 'SS'l allows reconstruction of a par- 
ticle track through the array [ioj . 

In particular, a charged current interaction will pro- 
duce a muon that can traverse the entire detector. This 
track-like topology allows reconstruction of the original 
neutrino direction to within a few degrees. An estimate 
of the energy of the muon is possible by measuring its 
energy loss, but this is complicated by stochastic losses, 
and in any case is only a lower bound on the original 
neutrino energy. 



B. Simulation 

In order to meaningfully compare our data with expec- 
tations from various signal hypotheses, we must have a 
detailed simulation of the atmospheric neutrinos and the 
subsequent detector response. For the input atmospheric 
muon neutrino spectrum, we generate an iso trop ic power- 
law flux with the NUSIM neutrino simulator [4^| and then 
reweight the events to standard flux predictions [H, . 
We have extended the predicted fluxes to the TeV energy 
range by fitting the low-energy region with the Gaisser 
parametrization [47j and then extrapolating above 700 
GeV. We add standard oscillations and/or non-standard 



flavor changes by weighting the events with the muon 
neutrino survival probability in Eqs. [TJ [21 or [51 

Muon propagation and energy loss near and within 
the detector is simulated using MMC ^4§\. Photon prop- 
agation through the ice, including scattering and ab- 
sorption, is modeled with photonics [49], incorporating 
the depth-dependent characteristic dust layers |39l]. The 
AMASIM program [s^] simulates the detector response, 
and identical reconstruction methods are performed on 
data and simulation. Cosmic ray background rejection is 
verified at all but the highest quality levels by a parallel 
simulation chain fed with atmospheric muons from COR- 
SIKA [5l|, although when reaching contamination levels 
of 0(1%) — a rejection factor of 10® — computational 
limitations become prohibitive. 

C. Atmospheric Neutrino Event Selection 

Even with kilometers of ice as an overburden, atmo- 
spheric muon events dominate over neutrino events by 
a factor of about 10^. Selecting only "up-going" muons 
allows us to reject the large background of atmospheric 
muons, using the Earth as a filter to screen out every- 
thing but neutrinos. In practice, we must also use other 
observables indicating the quality of the muon directional 
reconstruction, in order to eliminate mis-reconstructed 
atmospheric muon events. 

Our data sample consists of 1.3 x 10^" events collected 
with AMANDA-II during the years 2000 to 2006. The 
primary trigger for this analysis is a multiplicity con- 
dition requiring 24 OMs to exceed their discriminator 
threshold (a "hit") within a sliding window of 2.5 fis. 
As part of the initial data cleaning, periods of unstable 
detector operation are discarded, such as during the aus- 
tral summer months when upgrades and conflguration 
changes occur. After accounting for inherent detector 
deadtime in the trigger and readout electronics, the sam- 
ple represents 1387 days of livetime. During the data 
filtering, dead or unstable OMs are removed, resulting in 
approximately 540 modules for use in this analysis. Iso- 
lated noise hits and hits caused by electrical cross-talk 
are also removed [ioj. 

As a starting point for neutrino selection, we utilize 
the quality selection criteria from the AMANDA-II 5- 
year point source analysis [4l|. These cuts, not specif- 
ically optimized for high energy neutrinos, are efficient 
at selection of atmospheric neutrinos and achieve a pu- 
rity level of ^ 95%, estimated by tightening the quality 
cuts until the ratio between data and atmospheric neu- 
trino simulation stabilizes. The primary reconstruction 
and/or quality variables used in this selection are: 

1. the reconstructed zenith angle as obtained from a 
32-iteration unbiased likelihood (UL) fit; 

2. the smoothness, a topological parameter describing 
the homogeneity of the photon hits along the UL 
fit track; 
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3. the estimated angular resolution of the UL fit, using 
the width of the likelihood minimum [42i] ; 

4. the likelihood ratio between the UL fit and 
a Bayesian likelihood (BL) fit [43], obtained 
by weighting the likelihood with a zenith-angle- 
dependent prior. This weight constrains the track 
hypothesis to reconstruct the event as a "down- 
going" atmospheric muon. 

The strength of the smoothness and the likelihood ratio 
cuts also vary with the reconstructed zenith angle, as in 
general the cuts must be stronger near the horizon where 
background contamination is worse. Further discussion 
of the background rejection of these quality variables can 
be found in the point source analysis using these data 

a- 

To this selection we add further criteria to remove 
the final few percent of mis-reconstructed atmospheric 
muons. Specifically, we remove events with poor values 
in the following quality variables: 

1 . the space-angle difference between the UL fit track 
and the fit track by JAMS (a fast pattern-matching 
reconstruction; see Ref. pl|); 

2. the number of hits from direct (unscattered) pho- 
tons based on the UL fit hypothesis; 

3. the maximum length along the reconstructed track 
between direct photon hits. 

These selection criteria, as well as the analysis procedure 
described in section HVl were designed in a blind manner, 
in order to avoid biasing the results. Specifically, our ob- 
servables (the zenith angle and number of OMs hit, A'ch; 
see section HVAp were kept hidden when designing both. 
However, after unblinding, we found a small excess of 
high energy events above atmospheric neutrino predic- 
tions (444 events with 60 < iVch < 120 on an expecta- 
tion of ~350). While this is a relatively small fraction 
of the overall sample, and an excess at high iVch cannot 
be misinterpreted as one of our new physics hypotheses, 
a concentration of high energy background events could 
falsely suggest an atmospheric neutrino spectrum much 
harder than expected. 

We find that these events exhibit characteristics of mis- 
reconstructed atmospheric muons: poor reconstructed 
angular resolution; poor UL-to-BL likelihood ratio; and 
low numbers of unscattered photon hits based on the fit 
hypothesis. As atmospheric neutrino events show better 
angular resolution and likelihood ratio at higher ener- 
gies, we chose to revise our selection criteria to tighten 
the cuts on space-angle difference and angular resolution 
as function of the number of OMs hit, N^h- In partic- 
ular, from iVch = 50 to Nch = 80, we linearly decrease 
(strengthen) the required angular resolution and space- 
angle difference. These additional cuts were only applied 
to events with likelihood ratio lower than the median for 



a given zenith angle, as determined by atmospheric neu- 
trino simulation. We estimate that the purity of the final 
event sample is greater than 99%. 

D. Final Neutrino Sample 

After all selection criteria are applied, we are left with 
a sample of 5544 atmospheric neutrino candidate events 
with reconstructed zenith angles below the horizon -'^ . We 
may characterize the total efficiency of neutrino detec- 
tion, including all detector and cut efficiencies as well as 
effects such as earth absorption, via the neutrino effective 
area A'^g{E^,6, cj)), defined such that 

iVevcnts = J dE, dn dt '^^fj^"'^^^^ A:giE,,0,cj,) (10) 

for a differential neutrino flux d^/dE^dfl. Fig. [2] shows 
the and effective areas as a function of neutrino 
energy for event sample used in this analysis, as derived 
from the simulation chain described in the previous sec- 
tion. We have averaged over the detector azimuth (j>. The 
differences in effective area at various zenith angles are 
due to detector geometry. Earth absorption at high en- 
ergies, and the strong quality cuts near the horizon; the 
different effective areas for i^^ and are due to their 
different interaction cross sections. 

The simulated energy response to the Barr et al. at- 
mospheric neutrino flux ^46| (without any new physics) 
is shown in Fig. [3) For this flux, the simulated median 
energy of the final event sample is 640 GeV, and the 5%- 
95% range is 105 GeV to 8.9 TeV. 

IV. ANALYSIS METHODOLOGY 

A. Observables 

As described in section |TT1 the signature of a flavor- 
changing new physics effect such as VLI or QD is a deficit 
of events at the highest energies and longest baselines 
(i.e., near the vertical direction). For our directional ob- 
servable, we use the cosine of the reconstructed zenith 
angle as given by the UL fit, cos6'ul (with —1 being the 
vertical up-going direction). We use the number of OMs 
(or channels) hit, A'ch, as an energy-correlated observ- 
able. Fig. 0] shows the neutrino energy as a function of 
the simulated A^ch response. Fig. [5] shows the simulated 
effects of QD and VLI on both the zenith angle and Nch 
distributions, a deficit of events at high Nch and towards 
more vertical directions. Because the A^ch energy esti- 
mation is approximate, the VLI oscillation minima are 



A table of the atmospheric neutrino events is available at 
http: //www. icecube .wise . edu/science/data . 
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FIG. 2: Simulated detector effective area versus neutrino energy at the final analysis level. Left: effective areas for several 
zenith angle ranges. Right: zenith-angle-averaged effective areas for i/^ (solid) and (dotted). 
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FIG. 3: Simulated Vfi + energy distribution of the final 
event sample, assuming the Barr et al. input spectrum. 



FIG. 4: Simulated profile histogram of median neutrino en- 
ergy versus number of OMs hit (Nch), both for all zenith 
angles below the horizon and for various zenith angle ranges. 
Error bars on the all-angle points represent the ±lcr spread 
at each A'^ch. 



smeared out, and the two effects look similar in the ob- 
servables. Furthermore, the observable minima are not 
exactly in the vertical direction because the TVch-energy 
relationship varies with zenith angle (see Fig. [4]), since the 
detector is taller than it is wide. However, this geometry 
is beneficial for angular reconstruction of near-vertical 
events and so is still well-suited to this analysis. 



B. Statistical Methods 

To test the compatibility of our measured atmospheric 
neutrino (cos ^ul, -^ch) distribution with the various hy- 
potheses characterized by the VLI and QD parameters, 
we turn to the frequentist approach of Feldman and 
Cousins [13 • Specifically, we iterate over our physics 
parameters 9r, and our test statistic at each point in the 
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FIG. 5: Ratio of the simulated number of events given VLI (left, with n — 1, sm2^ = 1, and AS — 10 and QD (right, with 
n — 2 and D* = 10^^" GeV^^) to conventional oscillation predictions for the zenith angle and A'^ch distribution. 



parameter space is the log likelihood ratio comparing this 
to the best- fit point 9r, 



the hypothesis the best at the point Or- In other words, 
we marginahze over 9s in both the numerator and the 
denominator of the hkelihood ratio: 



AC^Or) = C{9r)~C{0r) (11) 

= -2\nP{{n,}\er) + 2lnP{{n,}\er) 



N 



i=l 



2 [ /it - + Hi In 



for binned distributions of observables with Ui counts in 
the ith bin, with fJ^iifii) expected given physics parame- 
ters 9r{dr)- For example, in a search for VLI effects, our 
physics parameters 9^ are the VLI parameters logj^Q A<5 
and sin 2^; a binned distribution of simulated iVch and 
cos 0UL gives us for a particular value of the VLI pa- 
rameters; and the distribution of A^ch and cos 0ul for the 
data gives us n^. 

As in Ref. [12], we characterize the spread in the test 
statistic A£ expected from statistical variations by gen- 
erating a number of simulated experiments at each point 
9r- To define the allowed region of parameter space 
at a confidence level (CL) a, we find the critical value 
A/Icrit(^r) for which a fraction a of the experiments at 
9r satisfy AC < ACcHt- Then our acceptance region 
at this CL is the set of parameter space {9r} where 

ACdata{Sr) < ACait{dr)- 

The above procedure does not a priori incorporate any 
systematic errors (or in statistical terms, nuisance param- 
eters). For a review of recent approaches to this problem, 
see [53| . We use an approximation for the likelihood ratio 
that, in a sense, uses the worst-case values for the nui- 
sance parameters 9s — the values that make the data fit 



ACp{er) = c{er,9s)-c{er,es) , 



(12) 



where we have globally minimized the second term, and 
we have conditionally minimized the first term, keeping 

9r fixed but varying the nuisance parameters to find 9s- 
This test statistic is known as the profile likelihood [541] . 
The profile likelihood is used in combination with a 



approximation in the minos method in minuit 55] 



56 



and is also explored in some detail by Rolke et 
Isrj . To extend our frequentist construction to the profile 
likelihood, we follow the profile construction method (ssl . 

we perform simulated experiments as before, but 
instead of iterating through the entire {9r,9s) space, at 
each point in the physics parameter space 9r we fix 9s to 

its best- fit value from the data, 9s. Then we recalculate 
the profile likelihood for the experiment as defined in 
Eq. [T21 As before, this gives us a set of likelihood ratios 
{A>Cp} with which we can define the critical value for a 
confidence level that depends only on 9r. 



C. Systematic Errors 

Each nuisance parameter added to the likelihood test 
statistic increases the dimensionality of the space we 
must search for the minimum; therefore, to add system- 
atic errors we group by their effect on the (cos6'ul, A^ch) 
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distribution. We define the following four classes of er- 
rors: 1) normalization errors, affecting only the total 
event count; 2) slope errors, affecting the energy spec- 
trum of the neutrino events and thus the Nch distribu- 
tion; 3) tilt errors, affecting the cos ^ul distribution; and 
4) OM efficiency errors, which affect the probability of 
photon detection and change both the cos6'ul and A'ch 
distribution. These errors are incorporated into the sim- 
ulation as follows: 

• Normalization errors are incorporated via a uni- 
form weight 1 ± -y/ {af -l- a|); 

• slope errors are incorporated via an energy- 
dependent event weight (i?/£'mcdian)'^''', where 
^'median is the median neutrino energy at the final 
cut level, 640 GeV; 

• tilt errors are incorporated by linearly tilting the 
cos 9uh distribution via a factor 1 -I- 2k (cos 6'ul + ^ ) ; 

• and OM efficiency errors are incorporated by re- 
generating atmospheric neutrino simulation while 
changing the efficiency of all OMs in the detector 
simulation from the nominal value by a factor 1-l-e. 

We split the normalization error into two components, 
ai and a2, to facilitate the determination of the conven- 
tional atmospheric flux, as we discuss later. 

Table H] summarizes sources of systematic error and the 
class of each error. The total normalization errors ai and 
a2 are obtained by adding the individual normalization 
errors in quadrature, while the tilt k and slope change A7 
are added linearly. Asymmetric error totals are conserva- 
tively assumed to be symmetric, using whichever devia- 
tion from the nominal is largest. Each class of error maps 
to one dimension in the likelihood space, so for example in 
the VLI case, £{9r,9s) = £(A(5, sin2^, a, A7, k, e). Dur- 
ing minimization, each nuisance parameter is allowed to 
vary freely within the range allowed around its nominal 
value, with each point in the likelihood space giving a 
specific prediction for the observables, A^ch and cos^ul- 
In most cases, the nominal value of a nuisance parameter 
corresponds to the predictions of the Barr et al. flux, 
with best-known inputs to the detector simulation chain. 

One of the largest sources of systematic error is the 
overall normalization of the atmospheric neutrino flux. 
While the total + simulated event rate for recent 
models [H, only differs by ±7%, this masks signifi- 
cantly larger differences in the individual and rates. 
We take the latter difference of ±18% to be more repre- 
sentative of the true uncertainties in the models. This 
is also in line with the total uncertainty in the flux esti- 
mated in Ref. fT3|. 

Another large source of error in the event rate arises 
from uncertainties in our simulation of the neutrino in- 
teractions, including the neutrino-nucleon cross section, 
parton distribution functions, and the neutrino-muon 
scattering angle. We quantify this by comparing our 
NUSIM simulation with a sample generated with the ANIS 



TABLE I: Systematic errors in the atmospheric muon neu- 
trino flux, separated by efi'ect on the observables cos and 
A'^ch (see section HV CI for details on the parameters). 



Error 


Class 


Magnitude 


Atm. Ufj, + Ufj, flux 


Ql 


±18% 


Neutrino interaction 


Ct2 


±8% 


Reconstruction bias 


a.2 


-4% 


i^T-induced muons 


02 


±2% 


Background contamination 




±1% 


Charmed meson contribution 


012 


±1% 


Timing residual uncertainty 


a2 


±2% 


Muon energy loss 


0L2 


±1% 


Primary CR slope (H, He) 


A7 


±0.03 


Charmed meson contribution 


A7 


±0.05 


Pion/kaon ratio 




±0.01/-0.03 


Charmed meson contribution 


K 


-0.03 


OM efliciency, ice 


e 


±10% 



simulator [g^I- ANIS uses the CTEQ5 cross sections and 
parton distribution functions i61|] , compared to MRS [i^] 
in NUSIM, and it also accurately simulates the neutrino- 
muon scattering angle. We flnd an 8% difference in the 
normalization for an atmospheric neutrino spectrum. 

A third signiflcant source of error is the uncertainty in 
the efficiency of the optical modules, that is, the proba- 
bility an OM will detect a Cherenkov photon. This has 
a large effect on both the overall detector event rate (a 
decrease of 1% in efficiency results in a decrease of 2.5% 
in event rate) and the shape of the zenith angle and N^h 
distributions. We quantify the uncertainty by compar- 
ing the trigger rate of down-going muons with simulation 
predictions given various OM efficiencies, including the 
uncertainty of hadronic interactions by usin g CO RSIKA 
air shower simulations with the SIBYLL 2.1 [6^, EPOS 
1.60 [M], and QGSJET-ii-03 [65] interaction models. We 
find that we can constrain the optical module efficiency 
to within ±10%/— 7%, consistent with the range of un- 
certainty determined in Ref. [41]. Furthermore, because 
uncertainties in the ice properties have similar effects on 
our observables, we model OM efficiency and ice scat- 
tering/absorption together as a single source of error of 
±10% (in efficiency). 

Other smaller sources of error were quantified with 
dedicated simulation studies or, if directly applicable to 
this analysis, taken from Ref. [4l|. For example, we de- 
termine the effect of a large contribution of "prompt" 
v^j^ from charmed particle decay by simulating the op- 
timistic Naumov RQPM flux [6^], and flnd that its ef- 
fects can be modeled with the normalization, slope, and 
tilt errors as shown in table HI Finally, we characterize 
our uncertainty in our reconstruction quality parameters 
( "reconstruction bias" in table [l| by investigating how 
systematic disagreements between data and simulation 
affect the number of events surviving to the flnal cut 
level. 
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D. Binning and Analysis Parameters 

In general, finer binning provides higher sensitivity 
with a hkehhood analysis, and indeed we find a mono- 
tonic increase in sensitivity to VLI effects while increas- 
ing the number of bins in cos^ul and Nch- However, 
because the further gains in sensitivity are minimal with 
binning finer than lOx 10, we limit ourselves to this size in 
order to avoid any systematic artifacts that might show 
up were we to bin, say, finer than our angular resolu- 
tion. We also limit the N^h range for the analysis to 
20 < A^ch < 120. While the multiplicity trigger requires 
24 or more OMs in an event, the hit-cleaning algorithms 
reduce the effective threshold to Nch ~ 20. We limit 
the high energy range to events with A'ch < 120 in or- 
der to avoid regions with poor statistics. This limits the 
possibility that a few remaining background events con- 
centrated at high energy might bias the analysis, which 
assumes the data can be modeled by atmospheric neu- 
trino simulation with a small eneTgy-independent back- 
ground contamination. The choice of Nch range reduces 
the number of candidate neutrino events in the analysis 
region to 5511. These binning choices were made in a 
blind manner, using simulation to determine sensitivity 
to the new physics effects. 

We also make a few more simplifications to reduce the 
dimensionality of the likelihood space. First, the phase rj 
in the VLI survival probability (Eq. [J) is only relevant if 
the VLI effects are large enough to overlap in energy with 
conventional oscillations (i.e., below ~100 GeV). Since 
our neutrino sample is largely outside this range, we set 
cos 77 = for this search. This means we can also limit 
the VLI mixing angle to the range < sin 2^ < 1. Sec- 
ond, in the QD case, we vary the decoherence parameters 
D* in pairs {D*,D*) and {D*,D^). If we set D* and D* 
to zero, after decoherence 1/2 of remain; with Dg 
and Z?7 set to zero, 5/6 remain; and with all D* equal 
and nonzero, 1/3 remain after decoherence. These lim- 
iting behaviors are relevant when considering sensitivity 
to different parts of the parameter space. 

Finally, in the absence of new physics, we can use the 
same methodology to determine the conventional atmo- 
spheric neutrino flux. In this case, the nuisance param- 
eters ai (the uncertainty on the atmospheric neutrino 
flux normalization) and A7 (the change in spectral slope 
relative to the input model) become our physics param- 
eters. The determination of an input energy spectrum 
by using a set of model curves with a limited number of 
parameters is commonly known as forward-folding (see 
e.g. Ref. [13). 

Table HIl summarizes the likelihood parameters used for 
the VLI, QD, and conventional analyses. 



V. RESULTS 

After performing the likelihood analysis on the 
(cos ^UL, A'ch) distribution, we flnd no evidence for VLI- 



TABLE II: Physics parameters and nuisance parameters used 
in each of the likelihood analyses (VLI, QD, and conven- 
tional) . 



Analysis 


Physics parameters 


Nuisance parameters 


VLI 

QD 
Conv. 


A(5, sin 2^ 

^3,81 ^6,7 
ai, A7 


ai, Q2, A7, K, e 

Qfl, Q2, A7, K, e 

Q2, K, e 



induced oscillations or quantum decoherence, and the 
data are consistent with expectations from atmospheric 
flux models. The reconstructed zenith angle and iVch dis- 
tributions compared to standard atmospheric neutrino 
models are shown in Fig. [SI projected into one dimension 
from the 10 x 10 two-dimensional analysis distribution 
and rebinned. Given the lack of evidence for new physics, 
we set upper limits on the VLI and QD parameters. 



A. Upper Limits on Violation of Lorentz Invariance 

The 90% CL upper limits on the VLI parameter A(5 for 
oscillations of various energy dependencies, with maximal 
mixing (sin 2^ =1) and phase cosry = 0, are presented in 
table Unl Allowed regions at 90%, 95%, and 99% confi- 
dence levels in the A5- sin 2^ plane for the rt = 1 hypoth- 
esis are shown in Fig. [71 The upper limit at maximal 
mixing of /S.5 < 2.8 x 10~^^ is competitive with that 
from a combined Super-Kamiokande and K2K analysis 

m- 

In the n — 1 case, recall that the VLI parameter AS 
corresponds to the splitting in velocity eigenstates Ac/c. 
Observations of ultra-high energy cosmic rays constrain 
VLI velocity splitting in other particle sectors, with the 
upper limit on proton-photon splitting of (cp — c)/c < 
lO^'^^ S]- While we probe a rather specific manifestation 
of VLI in the neutrino sector, our limits are orders of 
magnitude better than those obtained with other tests. 



B. Upper Limits on Quantum Decoherence 

The 90% CL upper limits on the decoherence parame- 
ters D* given various energy dependencies are also shown 
in table [ml Allowed regions at 90%, 95%, and 99% 
confidence levels in the Dg g,-D'Q 7 plane for the n — 2 
case are shown in Fig. [5] The 90% CL upper limit 
from this analysis with all D* equal for the n — 2 case, 
D* < 1.3 X 10-31 GeV-\ extends the previous best limit 
from Super-Kamiokande by nearly four orders of mag- 
nitude. Because of the strong E'^ energy dependence, 
AMANDA-II's extended energy reach allows much im- 
proved limits. 
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FIG. 6: Zenith angle and A'ch distribution of candidate atmospheric neutrino events in the final sample, compared with Barr 
et al. 46] and Honda et al. [l^l predictions (statistical error bars). 



TABLE III: 90% CL upper limits from this analysis on VLI 
and QD effects proportional to VLI upper limits are for 
the case of maximal mixing (sin 2^ — 1), and QD upper limits 
are for the case of — Dg — Dg = D7. 



-24 F 



n 


VLI (AS) 


QD (D*) 


Units 


1 


2.8 X 10"^'' 


1.2 X 10~^^ 




2 


2.7 X 10"^^ 


1.3 X 10"^^ 


GeV"' 


3 


1.9 X lO"^'^ 


6.3 X 10"^^ 


GeV-2 



C. Determination of Atmospheric Flux 

In the absence of evidence for violation of Lorentz in- 
variance or quantum decoherence, we interpret the atmo- 
spheric neutrino flux in the context of Standard Model 
physics only. We use the likelihood analysis to perform 
a two-parameter forward-folding of the atmospheric neu- 
trino flux to determine the normalization and any change 
in spectral index relative to existing models. As de- 
scribed in section HVDl we test hypotheses of the form 



E 



median 



A7 



(13) 



where d^id/dE is the differential Barr et al. or Honda 
et al. flux. 

The allowed regions in the Q!i-A7 parameter space are 
shown in Fig. [5] We display the band of allowed energy 
spectra in Fig. (TUl where we have constructed the al- 
lowed region by forming the envelope of the set of curves 




Sin^ 2f 



FIG. 7: 90%, 95%, and 99% CL allowed regions (from dark- 
est to lightest) for VLI- induced oscillation effects with n = 1. 
Note we plot sin^ 2^ to enhance the region of interest. Also 
shown are the Super-Kamiokande -I- K2K 90% contour [3| 
(dashed line), and the projected IceCube 10-year 90% sensi- 
tivity [68l ] (dotted line). 



allowed on the 90% contour in Fig. [9l The energy range 
of the band is the intersection of the 5%-95% regions of 
the allowed set of spectra, so restricted in order to limit 
the range of our constraints to an energy region in which 
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FIG. 8: 90%, 95%, and 99% CL allowed regions (from darkest 
to lightest) for QD effects with n — 2. 



AMANDA-II is sensitive. 

The central best-fit point is also shown in Figs. [5] and 
[TOl In fact, there is actually a range of best- fit points for 
the normalization, because of the degeneracy between the 
normalization parameter ai and the systematic error 0^2. 
Specifically, we find the best-fit spectra to be 



best-fit 



dE 



(1.1 ±0.1) 



E 



640 GeV 



0.056 



dE 



(14) 



for the energy range 120 GeV to 7.8 TeV, where the ±0.1 
is not the error on the fit but the range of possible best- 
fit values. This result is compatible with an analysis of 
Super-Kamiokande data [6§| as well as an unfolding of 
the Frejus data [t^, and extends the Super-Kamiokande 
measurement by nearly an order of magnitude in energy. 
Our data suggest an atmospheric neutrino spectrum with 
a slightly harder spectral slope and higher normalization 
than either the Barr et al. or Honda et al. model. The 
likelihood ratio AC of the unmodified Barr et al. spec- 
trum (at the point (0,1) in Fig.[9l) to the best-fit point is 
4.9, corresponding to the 98% CL. 



D. Discussion and Future Prospects 

To summarize, we have set stringent upper limits on 
both Lorentz violation and quantum decoherence effects 
in the neutrino sector, with a VLI upper limit at the 90% 
CL of A,5 = Ac/c < 2.8 x 10"^^ for VLI oscillations pro- 
portional to the neutrino energy E, and a QD upper limit 
at the 90% CL of D* < 1.3 x lO-^iQeV"^ for decoher- 
ence effects proportional to E^. We have also set upper 
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FIG. 9: 90%, 95%, and 99% aUowed regions (from darkest to 
lightest) for the normalization (l-|-ai) and change in spectral 
index (A7) of the conventional atmospheric neutrino flux, rel- 
ative to Barr et al. [4^. The star marks the central best-fit 
point. 



limits on VLI and QD effects with different energy de- 
pendencies. Finally, we have determined the atmospheric 
neutrino spectrum in the energy range from 120 GeV to 
7.8 TeV and find a best-fit result that is slightly higher 
in normalization and has a harder spectral slope than ei- 
ther the Barr et al. or Honda et al. model. This result 
is consistent with Super-Kamiokande data and extends 
that measurement by nearly an order of magnitude in 
energy. 

For an interpretation of the VLI and QD upper lim- 
its, we consider natural expectations for the values of 
such parameters. Given effects proportional to E^ and 
E\ one can argue via dimensional analysis that the new 
physics parameter should contain a power of the Planck 
mass Mpi or Mpj, respectively [7l|. For example, for the 
decoherence parameters D, we may expect 



D 



d* 



(15) 



PI 



for n > 2, and d* is a dimensionless quantity that is 
0(1) by naturalness. From the limits in table Hill we find 
d* < 1.6 X 10-12 {n = 2) and d* < 910 {n = 3). For 
the n = 2 case, the decoherence parameter is far below 
the natural expectation, suggesting either a stronger sup- 
pression than described, or that we have indeed probed 
beyond the Planck scale and found no decoherence of this 
type. 

While the AMANDA-II data acquisition system used 
in this analysis ceased taking data at the end of 2006, the 
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FIG. 10: Angle- averaged + T/f^ atmospheric neutrino flux 
(solid band, 90% CL from the forward- folding analysis) , mul- 
tiplied by E"^ to enhance features. The dotted line shows 
the central best-fit curve. Also shown is a previous result by 
Gonzalez-Garci'a et al. using Super-Kamiokande data as 
well as Barr et al. [4^ and Honda et al. (isj predictions. All 
fluxes are shown prior to oscillations. 



the neutrino sources detected. 
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next-generation, cubic-kilometer-scale IceCube detector 
has the potential to improve greatly upon the limits pre- 
sented here, as increased statistics of atmospheric neu- 
trinos at the highest energies probe smaller deviations 
from the Standard Model. In particular, IceCube should 
be sensitive to n = 1 VLI effects an order of magnitude 
smaller than the limits from this analysis [i^ (see also 
Fig. [7]) . We also note that we have also only tested one 
particular manifestation of VLI in the neutrino sector. 
A search of the atmospheric neutrino data for an un- 
expected directional dependence (for example, in right 
ascension) could probe other VLI effects, such as a uni- 
versal directional asymmetry (see e.g. |73j). 

Moving beyond searches with atmospheric neutrinos, 
once high energy astrophysical neutrinos are detected, 
analysis of the flavor ratio at Earth can probe VLI, QD, 
and CPT violation [tiI, [t^]. Another technique is to 
probe VLI via the potential time delays between photons 
and neutrinos from gamma-ray bursts (GRBs). Given 
the cosmological distances traversed, this delay could 
range from 1 /zs to 1 year, depending on the power of sup- 
pression by M-p\ 74]. Detection of high energy neutrinos 
from multiple GRBs at different redshifts would allow ei- 
ther confirmation of the delay hypothesis or allow limits 
below current levels by several orders of magnitude [TSj . 
Such a search is complicated by the low expected flux lev- 
els from individual GRBs, as well as uncertainty of any 
intrinsic 7 — 1/ delay due to production mechanisms in 
the source (for a further discussion, see Ref. [ill). Other 
probes of Planck-scale physics may be possible as well, 
but ultimately this will depend on the characteristics of 



APPENDIX: FORMALISM 

We present for the interested reader more detail of the 
phenomenological background to the atmospheric sur- 
vival probabilities for the VLI and QD hypotheses that 
we test in this work. 



1. Violation of Lorentz Invariance 

The Standard Model Extension (SME) provides an ef- 
fective field-theoretic approach to violation of Lorentz 
invariance (VLI) [73|. The "minimal" SME adds 
all coordinate-independent renormalizable Lorentz- and 
CPT-violating terms to the Standard Model Lagrangian. 
Even when restricted to first order effects in the neutrino 
sector, the SME results in numerous potentially observ- 
able effects [zl, [tI, ll^l . To specify one particular model 
that leads to alternative oscillations at high energy, we 
consider only the Lorentz-violating Lagrangian term 

\^{cL)^.uabLal'''D''L^, (A.l) 

with the VLI parametrized by the dimensionless coeffi- 
cient cl [zl]- La and L{, are left-handed neutrino dou- 
blets with indices running over the generations e, /i, and 

< — > 

T, and is the covariant derivative with A D B = 
AD^B - {D''A)B. 

We restrict ourselves to rotationally invariant scenar- 
ios with only nonzero time components in Ci, and we 
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consider only a two-flavor system. The eigenstates of the 
resulting 2x2 matrix correspond to differing max- 
imal attainable velocity (MAV) eigenstates. These may 
be distinct from either the flavor or mass eigenstates. 
Any difference Ac in the eigenvalues will result in neu- 
trino oscillations. The above construction is equivalent 
to a modified dispersion relationship of the form 

E'=p'cl + m^ct (A.2) 

where is the MAV for a particular eigenstate, and in 
general Ca ^ c [1, 01- Given that the mass is neghgible, 
the energy difference between two MAV eigenstates is 
equal to the VLI parameter Ac/c = [cai — Ca2)/c, where 
c is the canonical speed of light. 

The effective Hamiltonian H± representing the energy 
shifts from both mass-induced and VLI oscillations can 
be written [l^ 

_Am2 /I Q\ . ^cE f-1 0\ t 
^^-^^'[ l)^'^ + — 2^^[ ljU« 

(A.3) 

with two mixing angles 6 and ^. The associated 2x2 
mixing matrices are 

f cos9 sin6'\ /. 

and 

U, = ( ^^"^^^"') (A.5) 

^ y— sm4e+^ cos4 J ^ ' 

with 77 representing their relative phase. Solving the 
Louiville equation for time evolution of the state density 
matrix p, 

p=-i[iJ±,p] (A.6) 

results in the i/^ survival probability in Eq. [51 We refer 
the reader to Ref. [l^ for more detail. 

2. Quantum Decoherence 

Several constructions exist of a phenomenological 
framework for quantum decoherence effects [80| . A com- 



mon approach is to modify the time-evolution of the den- 
sity matrix p with a dissipative term $11 p: 



p = -i\H,p\^$Hp . (A.7) 

One method to model such an open system is via the 
technique of Lindblad quantum dynamical semigroups 
(81.] . Here we outline the approach in Ref. 32], to which 
we refer the reader for more detail. In this case we 
have a set of self-adjoint environmental operators Aj, and 
Eq. IA.7I becomes 



p^ -i{H,p\ + - Y^{{A,,pA,\^{A,p,A,\) . (A.8) 
i 

The hermiticity of the Aj ensures the monotonic increase 
of entropy, and in general, pure states will now evolve to 
mixed states. The irreversibility of this process implies 
CPT violation [80] . 

To obtain specific predictions for the neutrino sector, 
there are a gain several approaches for both two-flavor 
systems (36ll82j] and three- flavor systems (S^Hl]. Again, 
we follow the approach in [s^] for a three-flavor neutrino 
system including both decoherence and mass-induced os- 
cillations. The dissipative term in Eq. IA.8I is expanded 
in the Gell-Mann basis F^, ^ G {0, . . . , 8}, such that 



- Y,(\A,,pA,\ + {A,p, A,\) = L,.uP,.F, ■ (A.9) 

At this stage we must choose a form for the decoher- 
ence matrix and we select the weak-coupling limit 
in which L is diagonal, with Loo = and La = —Di,i G 
{1, . . . , 8}. The Di are in energy units, and their inverses 
represent the characteristic length scale(s) over which de- 
coherence effects occur. Solving this system for atmo- 
spheric neutrinos (where we neglect mass-induced oscil- 
lations other than — > Vr) results in the survival 
probability given in Eq. \E\ 

In Eq. [51 we must impose the condition Arri^/E > 
Dq — Di \ , but this is not an issue in the parameter space 
we explore in this analysis. If one wishes to ensure strong 
conditions such as complete positivity (s^ . there may 
be other inequalities that must be imposed (see e.g. the 
discussion in Ref. [SSi] ) . 



Jl] R. Gambini and J. Pullin, Phys. Rev. D 59, 124021 [3] V. A. Kostelecky and S. Samuel, Phys. Rev. D 39, 683 

(1999). (1989). 

[2] J. Madore, S. Schraml, P. Schupp, and J. Wess, [4] G. Amelino-Camelia, C. Lammerzahl, A. Macias, 

Eur. Phys. J. C 16, 161 (2000). and H. Miiller, Gravitation and Cosmology: AIP 



15 



Conf. Proc. 758, 30 (2005). 
[5] D. Mattingly, Living Rev. Relativity 8, 5 (2005). 
[6] S. Coleman and S. L. Glashow, Phys. Rev. D 59, 116008 

(1999). 

[7] S. L. Glashow, arXivihep-ph /0407087| 
[8] S. W. Hawking, Commun. Math. Phys. 87, 395 (1982). 
[9] V. A. Balkanov et al, Astropart. Phys. 12, 75 (1999). 
[10] E. Andres et al., Nature 410, 441 (2001). 
[11] J. A. Aguilar et al, Astropart. Phys. 26, 314 (2006). 
[12] J. Ahrens et al, Astropart. Phys. 20, 507 (2004). 
[13] M. Honda, T. Kajita, K. Kasahara, S. Midorikawa, and 

T. Sanuki, Phys. Rev. D 75, 043006 (2007). 
[14] G. D. Barr, S. Robbins, T. K. Gaisser, and T. Stanev, 

Phys. Rev. D 74, 094009 (2006). 
[15] Y. Ashie et al, Phys. Rev. Lett. 93, 101801 (2004). 
[16] M. Sanchez et ai, Phys. Rev. D 68, 113004 (2003). 
[17] M. Ambrisio et al, Phys. Lett. B 566, 35 (2003). 
[18] G. L. Fogh et ai, Phys. Rev. D 78, 033010 (2008). 
[19] M. C. Gonzalez-Garci'a and M. Mahoni, Phys. Rev. D 

70, 033010 (2004). 
[20] M. Mahoni, T. Schwetz, M. Tortola, and J. W. F. Valle, 

New Jour, of Phys. 6, 122 (2004). 
[21] G. Battistoni et ai, Phys. Lett. B 615, 14 (2005). 
[22] G. L. Fogli, E. Lisi, A. Marrone, and G. Scioscia, 

Phys. Rev. D 60, 053006 (1999). 
[23] M. H. Ahn et al, Phys. Rev. Lett. 90, 041801 (2003). 
[24] J. Ahrens and J. L. Kelley et ai, in Proc. of 30th ICRC 

(Merida), 2007; arXiv:0711.0353 
[25] D. Morgan, E. Winstanley, J. Brunner, and L. F. Thomp- 
son, Astropart. Phys. 29, 345 (2008). 
[26] J. Alfaro, H. A. Morales- Tecotl, and L. F. Urrutia, 

Phys. Rev. Lett. 84, 2318 (2000). 
[27] R. Brustein, D. Eichler, and S. Foffa, Phys. Rev. D 65, 

105006 (2002). 
[28] M. Gasperini, Phys. Rev. D 38, 2635 (1988). 
[29] A. Halprin and C. N. Leung, Phys. Rev. Lett. 67, 1833 

(1991). 

[30] G. Z. Adunas, E. Rodriquez-Milla, and D. V. Ahluwalia, 
Phys. Lett. B 485, 215 (2000). 

[31] J. Ellis, J. S. Hagelin, D. V. Nanopoulos, and M. Sred- 
nicki, Nucl. Phys. B 241, 381 (1984). 

[32] A. M. Gago, E. M. Santos, W. J. C. Teves, and 
R. Zukanovich Funchal, arXiv:hep-ph/0208166 

[33] G. Barenboim, N. E. Mavromatos, S. Sarkar, and 
A. Waldron-Lauda, Nucl. Phys. B 758, 90 (2006). 

[34] J. Ellis, N. E. Mavromatos, and D. V. Nanopou- 
los, Mod. Phys. Lett. A 12, 1759 (1997); J. Ellis, 
N. E. Mavromatos, D. V. Nanopoulos, and E. Winstan- 
ley, Mod. Phys. Lett. A 12, 243 (1997). 

[35] E. Lisi, A. Marrone, and D. Montanino, 
Phys. Rev. Lett. 85, 1166 (2000). 

[36] D. Morgan, E. Winstanley, J. Brunner, and L. Thomp- 
son, Astropart. Phys. 25, 311 (2006). 

[37] G. L. Fogh, E. Lisi, A. Marrone, and D. Montanino, 
Phys. Rev. D 67, 093006 (2003). 

[38] G. L. Fogli, E. Lisi, A. Marrone, D. Montanino, and 
A. Palazzo, Phys. Rev. D 76, 033006 (2007). 

[39] M. Ackermann et al, J. Geophys. Res. Ill, D 13203 
(2006). 

[40] J. Ahrens et al, Nucl. Inst. Meth. A 524, 169 (2004). 
[41] A. Achterberg et ai, Phys. Rev. D 75, 102001 (2007). 
[42] T. Neunhoffer, Astropart. Phys. 25, 220 (2006). 
[43] T. DeYoung et al, in Advanced Statistical Techniques in 
Particle Physics (Durham, U.K.), 235 (2002). 



[44] R. Abbasi et al, Phys. Rev. D 79, 062001 (2009). 

[45] G. C. Hill, Astropart. Phys. 6, 215 (1997). 

[46] G. D. Barr, T. K. Gaisser, P. Lipari, S. Robbins, and 
T. Stanev, Phys. Rev. D 70, 023006 (2004). 

[47] T. K. Gaisser, Cosmic Rays and Particle Physics (Cam- 
bridge University, U.K., 1991), p. 88. 

[48] D. Chirkin and W. Rhode, arXiv:he p-ph/0407075 

[49] J. Lundberg et ai, Nucl. lustrum. Meth. A 581, 619 
(2007). 

[50] S. Hundertmark, in Proc. of the Workshop on Simu- 
lation and Analysis Methods for Large Neutrino Tele- 
scopes, DESY-PROC-1999-01, 276 (1999); S. Hundert- 
mark, Ph.D. thesis, Humboldt Universitat zu Berlin 
(1999). 

[51] D. Heck et al, Tech. Rep. FZKA 6019, Forschungszen- 

trum Karlsruhe (1998). 
[52] G. J. Feldman and R. D. Cousins, Phys. Rev. D57, 3873 

(1998). 

[53] R. Cousins, in Statistical Problems in Particle 
Phystcs, Astrophysics and Cosmology: Proceedings 
of PHYSTAT05, edited by L. Lyons and M. Unel 
(Univ. of Oxford, U.K., 2005). 

[54] A. Stuart, K. Ord, and S. Arnold, Kendall's Advanced 
Theory of Statistics (Arnold, London, 1999), vol. 2A, 6th 
and earlier editions. 

[55] F. James and M. Roos, Comp. Phys. Comm. 10, 343 
(1975). 

[56] W. A. Rolke and A. M. Lopez, Nucl. lustrum. Meth. A 
458, 745 (2001). 

[57] W. A. Rolke, A. M. Lopez, and J. Conrad, in Sta- 
tistical Problems in Particle Physics, Astrophysics and 
Cosmology: Proceedings of PHYSTAT05, edited by 
L. Lyons an d M. Unel (Univ. of Oxford, U.K., 2005); 
|arXiv:physi cs/0403059 

[58] G. J. Feldman, "Multiple measurements and parameters 
in the unified approach," Workshop on Confidence Lim- 
its, Fermilab (2000). 

[59] K. Cranmer, in Statistical Problems in Particle 
Physics, Astrophysics and Cosmology: Proceedings of 
PHYSTAT05, edited by L. Lyons and M. Unel (Univ. of 
Oxford, U.K., 2005); arXiv:physics/0511028 

[60] M. Kowalski and A. Gazizov, Comp. Phys. Comm. 172 
3, 203 (2005). 

[61] H. L. Lai et al, Eur. Phys. J. C 12, 375 (2000). 

[62] A. D. Martin, R. G. Roberts, and W. J. Stirling, 
Phys. Lett. B 354, 155 (1995). 

[63] R. S. Fletcher, T. K. Gaisser, P. Lipari, and T. Stanev, 
Phys. Rev. D 50, 5710 (1994); R. Engel, T. K. Gaisser, 
P. Lipari, and T. Stanev, in Proc. of 26th ICRC (Salt 
Lake City) 1, 415 (1999). 

[64] K. Werner, F.-M. Liu, and T. Pierog, Phys. Rev. C 74, 
044902 (2006). 

[65] S. Ostapchenko, Phys. Lett. B 636, 40 (2006); 

S. Ostapchenko, Phys. Rev. D 74, 014026 (2006). 
[66] G. Fiorentini, A. Naumov, and F. L. Villante, 

Phys. Lett. B 510, 173 (2001); E. V. Bugaev et al, Nuovo 

Cimento C 12, 41 (1989). 
[67] S. Mizobuchi et al., in Proc. of 29th ICRC (Pune) 5, 323 

(2005); A. Djannati-Atai' et al, A&A 350, 17 (1999). 
[68] M. C. Gonzalez-Garci'a, F. Halzen, and M. Mahoni, 

Phys. Rev. D 71, 093010 (2005). 
[69] M. C. Gonzalez-Garci'a, M. Mahoni, and J. Rojo, JHEP 

0610, 075 (2006). 
[70] K. Daum, W. Rhode, P. Bareyre, and R. Barloutaud, 



16 



Z. Phys. C 66, 417 (1995). 
[71] L. A. Anchordoqui et al, Phys. Rev. D 72, 065019 
(2005). 

[72] D. Hooper, D. Morgan, and E. Winstanley, Phys. Rev. D 

72, 065009 (2005). 
[73] V. A. Kostelecky and M. Mewes, Phys. Rev. D 69, 016005 

(2004). 

[74] G. Amehno-CameUa, Intl. Jour, of Mod. Phys. D 12, 

1633 (2003). 

[75] U. .Jacob and T. Piran, Nature Phys. 3, 87 (2007). 

[76] M. C. Gonzalez-Garda and F. Halzen, JCAP 2, 008 



(2007). 

[77] D. CoUaday and V. A. Kostelecky, Phys. Rev. D 58, 
116002 (1998). 

[78] V. A. Kostelecky and M. Mewes, Phys. Rev. D 70, 

031902(R) (2004). 
[79] T. Katori, V. A. Kostelecky, and R. Tayloe, Phys. Rev. D 

74, 105009 (2006). 
[80] N. E. Mavrornatos, Lect. Notes Phys. 669, 245 (2005). 
[81] G. Lindblad, Commun. Math. Phys. 48, 119 (1976). 
[82] F. Benatti and R. Floreanini, JHEP 02, 032 (2000). 



